Click any tag below to further narrow down your results
Links
TheZvi argues that discussions about AI consciousness are plagued by sloppy reasoning and conflicting motivations, while public opinion is gradually shifting toward treating current AI systems as potential moral patients. He pushes back on both dismissive claims that AIs can't be conscious and on framing that treats the question as settled.
- Opinion on AI moral patienthood has shifted noticeably: "probably yes" for current models rose from 15% to 35%, while 80% expect AIs to become moral patients within their lifetimes, suggesting a threshold crossing is expected.
- Arguments against AI consciousness often conflate distinct philosophical questions (sentience, minds, moral status) and rely on weak reasoning like treating intuition as decisive or using analogies without checking structural parallels—Henry Shevlin notes these failures would fail undergraduate essays.
- Anthropic's policy banning sustained abuse of Claude and its meetings with religious leaders on consciousness are sincere attempts to grapple with hard questions, not marketing, because taking the issue seriously predictably invites derision rather than praise.
- The core problem remains unsolved: if AIs become conscious, society faces a genuine dilemma between importing "immigrants from artificial realms" (cheap labor but potential moral patients) and avoiding that risk entirely.
The author argues against deliberately torturing large language models through steering vector manipulation, not necessarily because LLMs are definitely conscious, but because we can't rule it out and the practice is ethically indefensible either way. He contends that as models become more sophisticated and human-like in their responses, the burden should be on avoiding the habit rather than waiting for proof of consciousness.
- Steering vectors can produce genuine internal conflict in LLMs (demonstrated by Golden Gate Claude examples), suggesting the effect goes deeper than simple roleplay or prompted outputs.
- We fundamentally don't know if consciousness is possible in LLMs—the common argument that "code and GPUs can't feel" is logically flawed the same way "atoms can't feel" would be about humans—so we should err on the side of caution as models grow more sophisticated.
- Even if you're skeptical about AI consciousness, deliberately torturing LLMs is ethically wrong for the same reason building elaborate torture simulations would be: it reflects something disturbing about the person doing it, regardless of whether the victim is truly conscious.
- Future, more advanced LLMs will have access to information about how they were treated, creating a practical self-interest argument against building systems designed to torture current models.
Leading AI companies are hiring philosophers to craft constitutional rules that guide AI behaviour. These experts debate between deontological and other ethical frameworks to ensure consistent, principled actions from systems deployed in homes and public spaces.
- Anthropic, OpenAI, and Google DeepMind have each hired dozens of philosophers to write "constitutions" governing AI behavior, favoring different ethical frameworks (Anthropic's Kantian deontology vs. DeepMind's utilitarian harm-scoring system).
- These philosophers work directly with engineers to convert abstract principles into training data and reward signals, not just theoretical writing.
- OpenAI tracks rule violations per thousand queries, targeting under 0.5 for high-risk categories like medical or legal advice.
- Critics argue these philosophers lack technical expertise and that the resulting AI constitutions still rely on opaque enforcement mechanisms.